The Energy Consumption of Transformer Fine-Tuning: A Roofline-Inspired Scaling Model

Transformer-based models underpin modern natural language processing but incur rapidly growing computational and energy costs. As training scales in both model size and parallelism, accurately predicting energy consumption has become critical for sustainable and cost-aware system design. We present …

View Post on Scour ↗

More from arXiv on Scour.